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Abstract 

Mean field type models describing the limiting behavior, as the number of players tends to 
+00, of stochastic differential game problems, have been recently introduced by J-M. Lasry 
and P-L. Lions. Numerical methods for the approximation of the stationary and evolutive 
versions of such models have been proposed by the authors in previous works . Convergence 
theorems for these methods are proved under various assumptions. 

MSG 2000: 65M06,65M12,91-08,91A23,49L25. 
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1 Introduction 

Mean field type models describing the asymptotic behavior of stochastic differential games (Nash 
equilibria) as the number of players tends to +00 have recently been introduced by J-M. Lasry 
and P-L. Lions |14 1 115 1 [16]. They may lead to systems of evolutive partial differential equations 
involving two unknown scalar functions: the density of the agents in a given state x, namely 
m = m{t,x) and the potential u = u{t,x). Since the present work is devoted to finite difference 
schemes, we will assume that the dimension of the state space is d = 2 (what follows could be 
generalized to any dimension d, although in practice, finite difference methods require too many 
computing resources when d > 4). In the periodic setting, typical such model comprises the 
following system of evolution partial differential equations 

du 

— {t,x)-uAu{t,x) + H{x,Vu{t,x)) = ^[m{t,-)]{x), in (0,T) x T^, (1) 

^(t,x) + z^Am(t,x) + div (^m(t,-)^(-,Vu(t,-))j (x) = 0, in(0,T)xT2, (2) 

with the initial and terminal conditions 

u{0,x) = uo{x), m{T,x) = mT{x), in T^, (3) 

given a cost function uq and a probability density rriT- 

Let us make some comments on the boundary value problem ([I])-®. 
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First, note that t is the remaining time to the horizon, (the physical time is in fact T — t), so 
uq should be seen as a final cost or incitation, whereas mr is the density of the agents at the 
beginning of the game. 

Here, we denote by = [0, 1]^ the 2— dimensional unit torus, hy v & nonnegative constant and 
by A, V and div, respectively, the Laplace, the gradient and the divergence operator acting on 
the X variable. The system also involves the scalar Hamiltonian H(x,p), which is assumed to 
be convex with respect to p and regular w.r.t. x and p. The notation ^{x, q) is used for the 
gradient of p i— )• H{x,p) at p = q. 
Finally, in the cost term <I>[m(t, ■)]{x), $ may be 

• either a local operator, i.e. ^[m{t, ■)]{x) = F[m{t, x)) where F is a regular function 
defined on R+. In this case, there are existence theorems of either classical (see [6]) or 
weak solutions (see [IS])! under suitable assumptions on the data, H and F. 

• or a non local operator which continuously maps the set of probability measures on 
(endowed with the weak * topology) to a bounded subset of Lip(T^), the Lipschitz functions 
on T^, and for example maps continously C^'"(T^) to C''+^'°(T^), for all A; G N and 
< a < 1. In this case, classical solutions of (HI)-® are shown to exist under natural 
assumptions on the data and some technical assumptions on H. 

We have chosen to focus on the case when the cost u\t=o depends directly on x. In some realistic 
situations, the final cost may depend on the density of the players, i.e. u\t=o = ^o[n^\t=o]{^)^ 
where $o is an operator acting on probability densities, which may be local or not. This case 
can be handled by the methods presented below, but we will not discuss it in the present work. 

System ([TJ- ([2|) consists of a forward Bellman equation coupled with a backward Fokker-Planck 
equation. The forward-backward structure is an important feature of this system, which makes 
it necessary to design new strategies for its mathematical analysis (see [151116)) and for numerical 
approximation. 

The following steady state version of ([I])- ([3]) arises when mean field games with infinite horizon 
are considered (ergodic problem): 

-iyAu{x) + H{x,Vu{x)) + X = ^m{-)]{x), in T^, (4) 

(dH \ 
m——(-,\7u) I (x) = 0, in T^. (5) 
dp J 

with the additional normalization of u: Jj2 u = 0. The unknowns in dH)-® are the density m, 
the function u and the scalar A. 

We refer to the mentioned papers of J-M. Lasry and P-L. Lions for analytical results concerning 
problems ([I])-® and dH)-® as well as for their interpretation in stochastic game theory. Let us 
only mention here that a very important feature of the mean field model above is that uniqueness 
and stability may be obtained under reasonable assumptions, see |131 [151 [16] , in contrast with 
the Nash system describing the individual behavior of each player, for which uniqueness hardly 
occurs. To be more precise, uniqueness for ([T])-® is true if $ is monotonous in the sense that 
for all probability measures m and m on T^, 

J ($[m](x) — ^[m]{x)){dm{x) — dm{x)) < ^ m = rh. (6) 

Examples of MFG models with applications in economics and social sciences are proposed in 
[12j . Many important aspects of the mathematical theory developed by J-M. Lasry and P-L. 
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Lions on MFG are not published in journals or books, but can be found in the videos of the 
lectures of P-L. Lions at College de France: see the web site of College de Prance, [15] . 
An important research activity is currently going on about approximation procedures of different 
types of mean field games models, see [13] for a numerical method based on the reformulation 
of the model as an optimal control problem for the Pokker-Planck equation with an application 
in economics and [9] for a work on discrete time, finite state space mean field games. We also 
refer to |101 lllj for a specific constructive approach for quadratic Hamiltonians. Finally, a semi- 
discrete approximation for a first order mean field games problem has been studied in [5]. 

In [2] and [1], the authors have proposed and studied finite difference methods basically re- 
lying on monotone approximations of the Hamiltonian and on a suitable weak formulation of 
the Pokker-Planck equation, both for infinite and finite horizon mean field games. These schemes 
were shown to have several important features: 

• existence and uniqueness for the discretized problems can be obtained by similar arguments 
as those used in the continuous case, 

• they are robust when v ^ (the deterministic limit of the models) , 

• bounds on the solutions (especially on the Lipschitz norm of u{t, •)), which are uniform in 
the grid step, may be proved under reasonable assumptions on the data. 

Fast algorithms for solving the discrete nonlinear systems arising in the discrete version of the 
MFC systems have been proposed in [3]. In the present paper, we would like to discuss the 
convergence of the schemes proposed in [2] in the reference case when H{x,p) is of the form 

H{x,p)='H{x) + \pf, (7) 

where /3 is a positive number greater than 1 and 'H is a periodic continuous function, under 
suitable monotony assumptions on the operator These assumptions lead to uniqueness for 
both the continuous and discrete system, and also to a priori and stability estimates under 
further assumptions. 

This work is organized as follows: in Section [21 we recall the finite difference schemes proposed 
in [21 [1] . Section [3] is devoted to basic facts concerning the discrete Hamiltonians and to the 
fundamental identity which leads to uniqueness and stability. When $ is a local operator, a 
consequence of this key identity is the a priori estimates on the solutions of the discrete MFG 
system that is presented in Section 15.11 Convergence theorems in the case when $ is a nonlocal 
smoothing operator are discussed in Section [H Finally, Section [5] contains convergence theorems 
in the case when <I> is a local operator. 

2 Finite difference schemes 

Let Nt be a positive integer and At = T/Nt, tn = nAt, n = 0, . . . , Nt- Let be a uniform grid 
on the torus with mesh step h, (assuming that 1/h is an integer N^), and Xij denote a generic 
point in T^. The values of u and m at {xij,tn) are respectively approximated by u'^j and m^j- 
Let (resp. m") be the vector containing the values u'^j (resp. mf^), for < i,j < 
indexed in the lexicographic order. For all grid function z on T|, all i and j, we agree that 

^ ^(i mod Nh),{j mod N^y 
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Elementary finite difference operators Let us introduce the elementary finite difference 
operators 

P^k. = ^^^^^V^ and p2+n),, = ^^^^^±ip^, (8) 
and define [Dhu]ij as the collection of the four possible one sided finite differences at Xij: 

[Dhu]ij = ((Z)i+n),j, (Z)i+n),_ij, {D+u)ij, iD+u)ij^i^ G R\ (9) 

We will also need the standard five point discrete Laplace operator 

Numerical Hamiltonian In order to approximate the term H{x,'S/u) in ([1]) or @, we con- 
sider a numerical Hamiltonian 17 : x ^ R, (x, qi,q2, Q'S; 94) 9 (^j 1i^Q2, Q3, Qi)- Hereafter 
we will often assume that the following conditions hold: 

(gi) monotonicity: g is nonincreasing with respect to qi and ^3 and nondecreasing with respect 
to q2 and 54. 

(g2) consistency: g {x,qi,qi,q2,q2) = H{x,q), Vx G T^, Vg' = (g'l, 92) S R^. 
(ga) differentiability: g is of class C^. 

(g4) convexity : {qi,q2, qs, qi) ^ 9 {x, gi, 92, qs, is convex. 
We will approximate H{-,'Vu){xij) by g{xij, [Dhu]ij). 

Standard examples of numerical Hamiltonians fulfilling these requirements are provided by Lax- 
Friedrichs or Godunov type schemes, see In this work, we focus on Hamiltonians of the form 
H{x,p) = T-L{x) + \p\^ , for which we choose 

g{x,q) =n{x) + G{q^,q:^,q^,ql), (10) 

where, for a real number r, r"^ = max(r, 0) and r~ = max(— r, 0) and where G : (R+)^ — > R+ is 
given by 

G{p) = \pf = {pI+pI+pI+pI)1 (11) 
Discrete version of the cost term ^[m{t, •)]{x) We introduce the compact and convex set 
ICh = {{m,j)o<i,j<Nh ■ h^'^'^ij = 1; > 0} (12) 

which can be viewed as the set of the discrete probability measures. 
We make the following assumptions, being local or not: 

{^hi) We assume that is continuous on }Ch- 

($/i2) The numerical cost is monotone in the following sense: 

{<^h[m] - <^h[m],'m - m)^ <0^^h[m] = ^/^[m], (13) 

where {u,v)2 = X]o<j j<Arh - This assumption and ((74) will be a sufficient condition 

for the discrete MFG system to have at most a solution, $/i being local or not. 
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If $ is a local operator, i.e. ^>[m](x) = F{ni{x)), F being a continuous function from to 
M, then the discrete version of ^> is naturally given by {^h[m])ij = F{mij). In this case, the 
operator is continuous on the set of nonnegative grid functions. 

If $ is a nonlocal operator, then we assume that the discrete operator has the following 
additional properties: 

i^hs) We assume that there exists a constant C independent of h such that for all times t and 
for all grid function m € JCh, 

\\^h[m]\\oo<C (14) 

and 

) (15) 
where d'f{x,y) is the distance between the two points x and y in the torus T^. 

($/i4) Define /C as the set of probability densities, i.e. nonnegative integrable functions m on 
such that fj2 m{x)dx = 1. For a grid function m/j G ICh, let rhh be the piecewise bilinear 
interpolation of rrih at the grid nodes: it is clear that rhh ^ We assume that there exists 
a continuous and bounded function uj : R-(_ — t- M-|_ such that u;(0) = and for all m € /C, 
for all sequences {mh)h, f^h ^ ^h-, 

II ^[m] - ^h[mh] |Il°o(t2) ^ ^ (11"^ - "^ft||Li(T2)) • (16) 
Let X/jTn be the grid function whose value at Xi^j is 

/ m{x)dx. 

J \x—Xi^j\oc<h/2 

It is clear that if ?ti G /C then X/jm € IC^ and that (|16p implies that 

lim sup II $[m] - ^>ft[2'ftm] \\Lo^n2^ = 0. (17) 



For example, if <I>[m] is defined as the solution w of the equation A^tD + w = m in T^, (A^ being 
the bilaplacian) , then one can define $/i[m/j] as the solution Wh of ^\wh + Wh = rrih in T^. It 
is possible to check that all the above properties are true. 



Discrete Bellman equation The discrete version of the Bellman equation is obtained by 
applying a semi-implicit Euler scheme to ([1]), 

- H^hu''^\j + ff(x,,„ [DhU^+\,) = i<^>h[m^])^,J, (18) 

for all points in and all n, < n < Nt, where all the discrete operators have been intro- 
duced above. Given (m")„=o,...,Arr-i, (fT8|l and the initial condition u-'j = uo{xij) for all (i,j) 
completely characterizes {u^)o<n<NT- 

Discrete Fokker-Planck equation In order to approximate equation ([2]), it is convenient to 
consider its weak formulation which involves in particular the term 

[ div ( 'm——(-,'Vu) \ (x)w(x) dx. 
Jt2 \ dp ) 
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By periodicity, 



r OH 

/ 'm(x)——(x,\/u(x))-\/w{x)dx 
Jj2 dp 



holds for any test function w. The right hand side in the identity above will be approximated 
by 

-h^'^mijVgg{xij, [Dhu\ij) ■ [Dhw]ij = h^'^Tij{u,m)wi, 
where the transport operator T is defined as follows: 



Tij{u,m 
( 



+mi+ij^(xj+ij, [Dhu]i+ij) - mij-^{xij, [Dhu]ij) 
\ oq2 oq2 



+ 



V 



d d 



The discrete version of equation ([2|) is chosen as follows: 



At 



+ u{Aj,m^)i,j + 7;jK+\m") = 0. 



(19) 



(20) 



This scheme is implicit w.r.t. to m and explicit w.r.t. u because the considered Fokker-Planck 
equation is backward. Given u this is a system of linear equations for m. It is easy to see that 
if satisfies ([20]) for < n < iVr and if m^T g /c^^ then m" € ICh for all n, < n < Nt- 

Remark 1 It is important to realize that the operator m i— > (— z^(A/im)jj- — 7ij{u,m)^ . ^ is the 
adjoint of the linearized version of the operator u i-^ (^—v{Afiu)ij + g{xij, [Dhu\ij)^ . 



Summary The fully discrete scheme for system ([I]), ([2]), ([3]) is therefore the following: for all 

< i,j < NhandO<n< Nt 



At 

^,3 



At 



with the initial and terminal conditions 



^ + u{Ahm%j + Tij (u"+i , m") =0, 



m. 



Nt 



/l2 



^ ? ■ j I no ^h/'2 



mT{x)dx, = uoixij), 0<i,j < Nh- 



(21) 



(22) 



The following theorem was proved in [2j (using essentially a Brouwer fixed point argument and 
estimates on the solutions of the discrete Bellman equation): 

Theorem 1 Assume that v > 0, that (gij-fgs) and (^hi) hold, that uq is a continuous function 
on and that tut G /C; then ^2l ]) - [2^l has a solution such that m"' € K-h, Vn. 
If furthermore 



(<^h3) holds 
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there exists a constant C such that 

r2 



< C(l + \qi\ + \q2\ + I93I + I94I) Vx G T , Vq'i , 52 , 93 , 94 



• no is Lipschitz continuous 
then maxo<n<Arj. (|ln"||oo + H-D/jU^Hoo) < c /or a constant c independent of h and At. 

Remark 2 Note that the technical condition on ^ is automatically true if g is given by 
ni\) and h is C^. 

Remark 3 A priori estimates in the case when ^ is a local operator will be given in § I5.il 

Since ([2T]l - ([22]l has exactly the same structure as the continuous problem ([II)-(l3]), uniqueness 
has been obtained in [2] with the same arguments as in |15) : 

Theorem 2 Assume that u > 0, that (gi)-(gi) and (^hi)-(^h2) hold, then ^2l \) -^2 ^) has a 
unique solution. 



3 Basic facts for numerical Hamiltonians of the form (1101)- (ITTI) 



We focus on numerical Hamitonians in the form ()10p - (|lip but obviously, what follows holds for 
g{x,q) = 'H{x) + cG{q), where G is given by ([TT]) and c is a positive constant. 
We use the following notations: for p G M^, \p\ (resp. |p|oo) is the Euclidean norm of p (resp. 
the max norm: \p\oo = niax,=i^...^4 \pi\). For a function p V'(p) G p G M^, tppi^p) G 
(resp. tpppip) G M"^^^) will stand for the gradient of ip (resp. the Hessian of tp). For a function 
{x,p) 'tp{x,p) G M, X G T2, p G M^, ipp{x,p) G is the gradient of p — > 'ip{x,p) and 
ippp{x,p) G M^^^ is the Hessian of p ^ ip{x,p). 

3.1 Basic lemmas 

Let us state a few lemmas about g and G: 
Lemma 1 For all p G (M+)^, Gp{p) = P\p\^~'^p and 

Gpp{p) = f3\pf-^h + /3(/3 - 2)b|^-4p ® p. (23) 
If then Gpp{p) — I3\p\^~'^l4 is a positive semi-definite matrix, which may be written 

Gpp{p) > f3\pf-^h- (24) 

// 1 < /3 < 2, then 

Gpp{p)>l3{P-l)\p\^~'h. (25) 
Lemma 2 For all q,q £ K^, let p,p £ (1R+)^ be given by 

P= {<h ,(lt)i <^nd P = (9r, 9^,93", 9^)- (26) 

We have that 

9{x, 9) - a{x, 9) - gq{x, q) ■{q-q)> G{p) - G{p) - Gp{p) ■ {p - p). (27) 
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Proof. See Appendix |Al ■ 

Lemma 3 If /3 >2, then for all q,q £ M^, for p,p £ given by ^ 



9{x,q) - g{x,q) - gq{x,q) ■ {q - q) > -^^maxQpf '^,\pf '^)\p-p\'^ (28) 

> (29) 



If 1 < (3 < 2, then for all q,q such that for p,p given by [26\) . p + p ^ 0, we have 

g{x, q) - g{x, q) - gg{x, q) ■ {q - q) > 2^~^f3{/3 - 1) min(|p|^-2, \pC^)\p - p\\ (30) 
Proof. See Appendix |Al ■ 

Lemma 4 For (3 > 2, there exists a positive constant c such that for all q,q,r € M'^, for all 
r? > 0, 

\{9q{x,q) -gq{x,q)) - rl < max{\pf-^ ,\pf-^) - + ??lrp^ . (31) 

where p,p £ (IR+)^ are given by [26\) . 
Proof. See Appendix El ■ 

3.2 A nonlinear functional Q{m,u,u) 

Let us define the nonlinear functional Q acting on grid functions by 

Q{m, u, u) 

Nt 

= E E {^{x,,, [DiJ%,) - g{x,„ [Du%,) - g,ix,„ [Du^],,) ■ {[Du^],, - [Du-]^,)) . 

n=l i,j 

(32) 

Under Assumption (g'4), it is clear that Q{m,u,u) > if m is a nonnegative grid function. If g 
is of the form ()10p - ()lip . we have a more precise estimate: 

Lemma 5 If m is a nonnegative grid function and if g is of the form lilO \ )-[Tl ]) with j3 > 2, 
then 

, Nt 

g{m,u,u) > ^^rnl-'maxi\pl/-\\p2jf-')\plj - p^f 



n=l I, J 



Nt 



(33) 



^ 2/^-2(Li) EEK7^bM-PM-r 

n=l i,j 



(34) 



where pfj and p^^ are the four dimensional vectors 

Pl^ = [[{Dtu-),,)- , ((Z?+n«)._i,,)+ , mu^.,,)- , {{Dtu-),,^,) 

Pl^ = [[{Dtun^,Jy , ((I?^n").-ia)^ , ((^2^^")m)" > {(Dtn^kj-i) 
Furthermore, if m is bounded from below by m, then 

g{m,u,u) > ^,^_3^ _ ^^\[Du^]i,, - [Z^n"],,, . (35) 

n=l j,7 
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Proof. We deduce (j33|) from (j28|) and (|29|1 . If m is bounded from below by m > 0, we deduce 
that 



Nt 

G{m,u,u) > - 

n=l ij 



Nt 4 

- ^ ^ ^ "^^'^^ - 

n=l fc=l 

Since for each < z, j < A'^^, each 1 < A; < 4, the quantity ([Dn"]jj)fc — ([Dn"]jj)fc 
at least once in the sum above, we deduce that 

Nt i 
' n=l i,j k=l 



appears 



Nt 

m 



n=l jj 



where the second inequality comes from the fact that for four positive numbers (0^)^=1 4 

Remark 4 The same kind of argument shows that if 1 < f3 < 2, and mfj > m, then 

Nt Nt 

n=l ij n=l ij 

Nt 4 TVr 4 

> 22/^-^/3(/3 - i)mEEEM,)>^\' ^ - i)-EEE 

n=l ij k=l n=l i,j k=l 

Nt 

>2^^-^l3i(3-l)rnY,Y^\[Du^],,f, 

n=l i,j 

where p"^ is given by (34\ )- 

3.3 A fundamental identity 

In this paragraph, we discuss a key identity which leads to the stability of the finite difference 
scheme under additional assumptions. Consider a perturbed system: 



'"^At + i^(Ahm")i J + 7-,,- (^2"+i , m") = ft^;, . 
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equation in (j20p . multiplying the result by u^j — u^j and summing for all n = 0, . . . , Nt — 1 



Multiplying the first equations in (|36|1 and (j2T|) by m"^ — m"^- and subtracting, then summing 
the results for all n = 0, . . . , Nt — 1 and all we obtain 

— - - (u" - -S"), (m" - m"))2 - zy(Afc(u"+^ - u"+^),m" - m")2 

n=0 
Nt-1 

+ E E(^^(^«.^-' [^^""^']^^) - 9ix^,J, [D,u-'-%))iml^ - rhl^) (37) 

ra=0 ij 

= ($/,[m"]-$;,[m"],m"-m")2- ^ (a", m" - m")2, 

n=0 n=0 

where (X, y)2 = Yli j Similarly, subtracting the second equation in from the second 

tiplying the result by r""*"-^ - ■'^""'"-^ 

and all (i, j) leads to 

Nt-1 . 

Y At (^"""^^ " " ("^"^^ " ^""""^^ " ^"'^^))2 + ^(("^" - 'Tl"), A/,(7X"+1 - 7i"+^))2 

n=0 
Nt-1 

- E E<.-[^'^(""^'-^"'"')]M-5'?KiJ^^^"+'].i) 

n=0 i,j 

72=0 ij 72=1 

(38) 

Adding ([37|) and (p8]) leads to the fundamental identity 

1 , TV ~ TVr A''t ~Nt\ , ^ r ~0 ~0\ 

Ar ^ Ar ^ 

A^T-l 

+ g{m,u,u)+G{rh,u,u)+ ^ (^^[m"] - $,,[m"], m" - m")2 ^3g^j 

n=0 

Nt-1 Nt 

= Y - ^")2 + E(^""'' - ^")2- 

Ti=0 n=l 

It is important to note that under assumptions ((74) and {^h2)-, the second line of PU|) is made of 
three nonnegative terms. This is the key observation leading to uniqueness for ()2ip -()22p. but it 
may also lead to a priori estimates or stability estimates under additional assumptions including 
for example the assumptions of Lemma [5j 

4 Study of the convergence in the case when $ is a nonlocal 
smoothing operator 

Hereafter the Hamiltonian is of the form ([7]). In all the following convergence results, we assume 
that system ([I])-© has a unique classical solution. This is always the case if $ is monotonous 
in the sense of ([6]) and continuously maps the set of probability measures on (endowed with 
the weak * topology) to a bounded subset of Lip(T2), the Lipschitz functions on T^, and for 
example maps continously C'='"(T2) to C''+^'"(T2), for all A; G N and < a < 1. 
We summarize the assumptions made in § H] as follows: 
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Standing assumptions (in Section [4]) 

• We take u > 

• The Hamiltonian is of the form ([7]) and the function x — )■ ^{{x) is on 

• the functions and rriT are smooth, and rriT € /C is bounded from below by a positive 
number 

• We assume that ^ is monotone in the sense of ([6]), nonlocal and smoothing, so that there 
is a unique classical solution {u,m) of ([II-(l3]) such that m > 

• We consider a numerical Hamiltonian given by ()10|l - ()lip and a numerical cost function 
such that {<^hi), ($fc2), (^fcs), and (^>m) hold. 



4.1 The case when /3 > 2 

Theorem 3 We make the standing assumptions stated at the beginning o/ § [7] and we choose 
/3>2. 

Let Uh (resp. rrih) be the piecewise trilinear function in C([0, T] xT^) obtained by interpolating the 
values u'^j (resp rn^-) at the nodes of the space-time grid. The functions converge uniformly 
and in L''(0, T; Ty^''^(T^)) to u as h and At tend to 0. The functions mfi converge to m in 
C'^{[0,T];L'^{T^))nL'^{0,T;H''{T^)) as h and At tend to 0. 

Proof. Note that mh{t, ■) e K for any t € [0,T]. 

We call tt" and m" the grid functions such that n"^ = u{nAt,Xij) and = Ih{m{tn, •)). The 
functions ■u" and are solutions of ()36p where a and b are consistency errors. From the fact 
that {u,m) is a classical solution of ([IJ-dS]), we infer from the consistency of the scheme (in 
particular from (fT7|l) that iiiaxQ<n<NT{.\W^\\L°°{Tl) + II^"IIl°°(t2)) tends to zero as h and At tend 
to zero. 



Step 1 As a consequence of the previous observations, the fundamental identity ([39|) holds, 
and from ()22p . can be written as follows: 



h^Atg{m, u, u) + h^Atgim, u, u) + h^At ^ («>h[m"] - - m")2 

(40) 

=h^At (a",m"-m")2 + /i^At^(6"-\?/"-u")2. 

n=0 n=l 

The a priori estimate on the discrete Lipschitz norm of given at the end of Theorem [1] implies 
that lim/j At^o ™ax„ |(6"~^,u"' — il'^)2\ = 0. From the fact that G JCh, we also get that 
liuifi^At^o max„ Ka", m" — m"')2| = 0. We can then use ([33]) to deduce that 

Nt-I 

At/.2 ^ ^mI:,.max(|p^;lr^|p-+^r2)|pI;;l-p^^^^ = 

(41) 

n=0 i,j 
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where p^j and p"^- are given by (fM|) . Since m"^- is bounded from below by a positive constant, 
we also deduce from ([55D that 



Nt 



h'AtY,^\[DhU^]ij - [Df,u^]ij = o(l). 



n=l 



(42) 



Step 2 We introduce e = m — m . Subtracting the second equation in p6p from the second 
equation in pT|) . multiplying the result by e|^- and summing for all ^ = 0, . . . , n — 1 and all (i, j) 
leads to 



n-l 



n-1 



n-l 

1=0 i,j 
n-l 

--h'AtY,ib',e%. 



This implies that 



n-l 



n-l 



(43) 



It is clear that 



h^At 



n-l 



o(l) h^At^We 



1 

n-l \ 2 

2 



£=0 



From Lemma SI we know that there exists an absolute constant c such that for all rj > 0, 

< rhl^ max(|pgT-^ iP^'r') (^Ipg' " ^^S'l' + v\[D,e%^^\ 



where Pi~j^,pl~j^ G (1^+)^ given by (|34p . Using the L°° bound on [DhU^^^] uniform w.r.t. i, 
h and At given in Theorem [H we obtain that there exists a constant c such that for all ij > 0, 
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+ v\[Dhe\/ 



(44) 
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We shall also use the standard estimate : 



(45) 



for a positive constant C independent of h. Finally, from ()43p . a very classical argument making 
use of dHD, dm) and ([12]), the L°° bound on [_D/jU^+i] uniform w.r.t. h and Ai, and the fact 
that /i||e^^||2 = 0(1), leads to the estimate: 



max hHe% + Mh^ V Ww^] = oil). 

0<i<NT ^ ir 2 



(46) 



We easily deduce from ()46p the claim on the convergence of rrih to m. 



Step 3 We have found that maxo<£<iVj, I/j(m(t^, ■))||| = o(l). From (<I*/s4)5 this implies 

that 



z.(A;,n"+i),j + [I?/,n"+^]ij) = ($[m(t„+i, ^^(xij) + o(l), 



,n+l 



At 



The uniform convergence of the piecewise linear functions (defined by interpolating the 
values u^j) to u is obtained from classical results on the approximation of Bellman equations by 
consistent and monotone schemes. From this and (j42p . we also deduce the convergence of to 
uin L^{0,T;W^^^{T'^)). m 



Remark 5 In the present case, additional assumptions on the order of the discrete scheme 
should lead to error estimates. We will not discuss this matter. 



4.2 The case when 1 < /? < 2 

Theorem 4 We make the standing assumptions stated at the beginning 0/ § [7] and we choose (3 
such 1 < /3 < 2. 

Let Uh (resp. m^) be the piecewise trilinear function in C([0, T] xT^) obtained by interpolating the 
values ufj (resp m^j) at the nodes of the space-time grid. The functions converge uniformly 
and in L^(0, T; VF-^'^(T^)) to u as h and At tend to 0. The functions converge to m in 
L2((0,r) X T^) as h and At tend to 0. 



Proof. 



Step 1 We start from (|40p where a and b are the same consistency errors (with the same 
bounds) as in the previous paragraph; using ([30|) . this implies that 

Nt-I 

e=o i,j 

(47) 

where Pi^^,Pi~j^ G (1^+)^ given by ()34p . From the a priori estimates on || [Dn^+^J ||^ and 
||[L''u^+^]||oo, the inequalities 1 < /3 < 2 and the bound from below on m, there exists a positive 
constant c such that 
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Hence 

Nt-1 

e=o 1,3 

It is easy to see from the periodicity that (08]) impUes 

Nt-I 2 

E =o{l)- (49) 

i=0 i,j 

Step 2: a priori estimate on (mfj) Multiplying the second equation in (|2ip by m'^j, 
summing with respect to the indices i and j and n, we get that 

n=0 i,j n=0 n=0 i,j 

(50) 

In a very classical way, using the continuity of gq and the a priori bound on \\DhU^'^^\\j;^oa(j2^ 
leads to the existence of a positive constant C such that 

max|lm"||i + At V 11 [D/,m"] 11^ < Cllm^^ ||i (51) 

n — ' 

n=0 

Remark 6 From \21\). we see that for all grid function (rij) on T^, 

Nt~1 _ n 

n=0 j,j 

=^ E E(V'^"^")m • (V.r"),,- + 5] 5]m^,.5,(x,,,[D,^"+i],,).[Z),r"],,,. 

n=0 n=0 ij' 

From the a priori hound on \\DhU^^^'^i^oc(j2^ and \51\) . we infer that 



(52) 



Z-/n=0 i,j 



sup ^^'^ ^'^ < C. 

Step 3 From ()5ip . we see that the family of functions {ruh) is bounded in L'^{0,T; H^{T'^)). 
Moreover, from (|52p and the a priori bound on ||D/in"+^||^oo(']p2^, we can use the same arguments 
as in e.g. [7] pages 855-858 and prove that the family of functions (m/j) has the following 
property: there exists a constant C such that 

Nt-1 / \ 
\\mhi--T,-)-mh{;-)\\har,T)xT^)<Crh''At ^ E I [^^^^mI' + EKj')' ' (^3) 

for all r G (0, T). Since the right hand side is bounded by Cr/i^||m^^ Hg, Kolmogorov's theorem 
(see e.g. [1], [8], page 833) implies that the family of functions (m/i) is relatively compact in 
L^((0, T) X T^): we can extract a subsequence of parameters h and At tending to such that 
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nih converges to m strongly in L^((0,T) x T^), and (I53p holds for m. 
Therefore, from (^m), 



Nt 



n=0 

On the other hand, from (|40p . we see that 



^.g™ 0^*^ II ^[^(in, •)] - = 0. 



Nt-1 

Ath^ ^ («>/,[m"] -$^[m"],m"-m'")2 = o(l). 

n=0 



Then, using ()17p . we deduce from the previous two formulas that 

Nt 

m At 

At 



lim AtY] {^m{tnr)]{x) - ^m{tn,-)]ix)) {m{tn,x) - m{tn,x))dx = 0, 



which implies that 

rT 



($[m(t, •)](x) - <I>[m(t, ■)\{x)) {m{t,x) - m(t,x))dxdt = 0, 

JTl 

The monotonicity of <I> then implies that m = rh. From the uniqueness of the limit m, we have 
proven that the whole family m/j converges to m in L^((0,T) x T^) as h and At tend to zero. 
We conclude as in Step 3 of the proof of Theorem [3] for the convergence of Uh to u in [0, T] x T^) 
and in L2(0,r;VFi'2(T2)). ■ 

5 Study of the convergence in the case when $ is a local operator 

5.1 A priori estimates for (I21l) -( l22|) with local operators $/i 

We have a result similar to Theorem 2.7 in |16| : 

Lemma 6 Assume that < mxix) < tTit and that uq is a continuous function. If g is given 
by [W\)-^^, {^h[m])Lj = F{mij), where 

(Fi) F is a function defined on [0, oo) 

(F2) there exist three constants 6 > and 7 > 1 and Ci > such that 

mF{m) > 5\F{m)p - Ci, Vm > 0, 

then there exists two constants c and C > such that 
• ufj > c, for all n, i and j 

Nt o Nt-1 

h'AtJ^Yl\^D,u^kj\ +h'At^Yl\^{ml^)\'' <C (54) 

n=l i,j n=0 i,j 

• 

max /i^y |u", | < C (55) 
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• Finally, let us call f/" the sum ^ u"^- and Uh the piecewise linear function obtained 

by interpolating the values U"' at the points (in); the family of functions (Uh) is bounded 
in W^'^{0,T) by a constant independent of h and At. 

Proof. 

From the two assumptions on F, we deduce that F = infmGR+ F{m) is a real number and 
that F_ = minm>o F{m). Note that F_ = F{Q) if F is nondecreasing. 
A standard comparison argument shows that 



> min u^ix) + [ F — maxT^fx) | t„ > min u^ix) —T\F — maxT^fx) ) , 



SO ufj is bounded from below by a constant independent of h and At. 
Consider u^j = nAtF{rhT) and m"^- = mj^ for all i,j,n. We have 



At 

m- ' —m; 



v{AhU^+^)ij + g{xi^j, [DhU 



F(mT)+^(xij), 
0. 



Identity ([39]) becomes 



"i,]) - F{mT){mlj -mr) 



h^AtG{rn,u,u) + h^Atg{m,u,u) + h^At ^ ^(F(m- 

n=0 jj' 

=/i2At ^ ^■H{xi,j){ml^ - rhlj) + h'^{m^^ - fhT,u^^ - T F{mT))2 - h^{ni^ - rhT,u%. 

(56) 

Note that 

^(m, u, u) = ^/(m, u, 0) and ^(m, u, u) = ^(m, 0, u), 
because tt"^ does not depend on On the other hand, 

1. Since the function % is bounded, and m" is a discrete probability density, there exists a 
constant C such that 



Nt-I 

h^At E^(^«.i)K. 

n=0 jj 



< C. 



2. Since m — tut is nonpositive with a bounded mass, and since is bounded from below 
by a constant, there exists a constant C such that 

h^{m^T _ fnT,u^^ -TF{mT))2 < C. 

3. Since uq is continuous on and TTiQ is a discrete probability density, there exists a constant 
C such that 

-/i2(m° -mT,u°)2 < C. 
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4. Finally, we know that 

{F{mlj) - F{mT)){mlj -mT)>6 \F{ml^\^ -Ci- ThTF{mlj) - mljF{mT) + fhTF{mT). 



Moreover, since 7 > 1, there exists two constants c = | and C such that 5 F{m^j) 

7 

— C. Since m" € /C/j, summing yields that for a possibly different 
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mTF{m^ ,j) > c 
constant C, 



Y.{F{ml^) - F{fnT))imlj - fuT) > ch' Yl " 



In the case /3 > 2, we get (|54|) from (|56|) . from (|35|) and from the four points above. In the case 
1 < /3 < 2, we get ()54p from (|56p . from Remark H] and from the four points above. 
Finally, summing the first equation in (j2ip for all i, j , < I < n one gets that 



g-l n— 1 

j,j f=0 i,i i=0 i,j i,j 



Using (|54|) . we get that there exists a constant C such that 

^'E<j-^^' 

and since n"^ is bounded from below by a constant, we get ([S^ . 

Finally, remember that [/" is the sum h'^ ^ u'^j ; summing the first equations in (|2ip for all 
we obtain that 

rrn+l _ ttu 

The a priori estimate (j54p implies that At X]^=o~^ is bounded by a constant. This implies 

that the piecewise linear function Uh obtained by interpolating the values f7" at the points (tn) 
is bounded in W^'^{0,T) by a constant independent of h and At. ■ 

5.2 Convergence theorems 

The case when $ is a local operator, i.e. <I>[m](a;) = F{m{x)) brings additional difficulties, 
because there is no a priori Lipschitz estimates on uy^: such estimates were used several times 
in the proofs of Theorems [3] and HI 

For simplicity, we are going to make the assumption that the continuous problem has a classical 
solution: existence of a classical solution can be true for local operators for example, it has 
been proved in ^ that if /3 = 2, and F is and bounded from below, and if the functions 
and rriT are then there is a classical solution. 

Remark 7 In the case of the stationary problem it can he proved that, if (F2) holds with 
^ > 2 (2 is the space dimension) and F is nondecreasing, then ^ has a classical solution for 
any /3 > 1, hy using the weak Bernstein method studied in I17f . 
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Standing assumptions (in § 15. 2p 

• We take u > 

• The Hamiltonian is of the form d?]) and the function x 'H(x) is on 

• the functions and mx are smooth, and uit € /C is bounded from below by a positive 



• We assume that (Fi) and (F2) hold that there exists three positive constants 5, ??i > 
and < ??2 < 1 such that F'{m) > 6mm(m^^,m~^^) 

• We consider a numerical Hamiltonian given by (jlOp - (jlip 

Theorem 5 We make the standing assumptions stated above and we assume furthermore that 
there is unique classical solution (u,m) of (CP-f^) such that m > 0. 

Let Uh (resp. nih) be the piecewise trilinear function in C([0, T] xT^) obtained by interpolating 
the values u"^- (resp m^-) at the nodes of the space-time grid. The functions Uh converge in 
L^(0, T; W^'^ir'^)) to u ash and Ai tend to 0. The functions ruh converge to m in L^"^^ (^(g, T) x 
T^) as h and At tend to 0. 

Proof. Call fh = maxm(t, x) and < m = minm(t, x). 

Step 1 We start from ()40p where a and h are the same consistency errors (with the same 

bounds) as in § m From Lemma O the a priori bound (j55p holds for Uh. This implies that 

liiii/i,At->o ™Eix„ Kfe"-"^, — ?i")2! = 0. From the fact that m" € Kh, we also get that 

limh,Ai^o max„ \{a'^,m'^ - m"')2| = 0. 

Therefore, if /3 > 2, we obtain (HTD and ()i2]l . 

If 1 < /3 < 2, we are going to prove that (|42p also holds: we have 



number 



Nt-I 



£=0 i,i 



where p^i^^ ^p^i^"^ G (1^+)^ are given by Let us define for brevity 




•if|pS^i<ipS 



then 
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If Ipg^l < bS^I and \^-^^\ > |pSV2, then 



1/3-3 



+ 



6/3- 



If = and / 0, then V^^^ > 2^-^/3(13 - 
From the observation above, and since mfj > m, 



;xi+l\2 



Nt-1 



1=0 i,j 



+1 



0(1). 



Note also that from the regularity of n, |pf j^^j'^ ^ is bounded from below by a constant indepen- 
dent of h, At, Thus 



Nt-1 

Ath^ E E 

^=0 ij 



/ 1 



'1|2 
o^+l|/3 



0(1). 



Using a Holder inequality, we deduce that 



Nt 



e=i ij 



and finally (fi2]) . 



Step 2 We also obtain from (00]) that 



Nt-1 



h'At E E(^K.) - ^K,)K. - ^m) = 

n=0 i,j 



We split the sum w.r.t. in the left hand side of ()57p into 

■JO 



Nt-1 

S2 = E E(« 

n=0 jj' 



J "^1,3 / 



F'{ml^+t{ml^-ml^))dt. 



(57) 
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Call m = ma,xm{t, x) and m = min?7i(t, x) > 0; there exists a positive number c depending on 
m and fh but independent of h and 6, and {i,j,n) such that 



^1 



The latter inequality comes from the nondecreasing character of the function x '■ [0) u] ^ 
x{z) = ^''^'^yZl^^^ ■ Thus, x{z) ^ x(0) = y^^ ■ Hence, there exists a constant c depending on 
the bounds on the density m solution of dH)-® but not on h and At, and {i,j,n) such that 



On the other hand 



But there exists a constant c such that for all y € [m, m] : if z > y + 1 

{z - yfz-^^ >{z- yf~^- inf ^ > c(z - 

and if y < < y + 1, 

Therefore there exists a constant c such that 



Then ()57p implies that 
Then, a Holder inequality leads to 

Nt-1 

lim /i^At V y Im",- - m"/-'?2 ^ g. 
n=0 t,j 
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Step 3 From the previous two steps, up to an extraction of a sequence, m/j min ((0, T) x 
T^) and almost everywhere in (0,r) x T^, Vu/j converges to Vu strongly in L^((0,r) x T^). 
Moreover, from the last point in Lemma El the sequence of piecewise linear functions (Uh) on 
[0,r] obtained by interpolating the values C/" = h'^ Yli j '^i'j points is bounded in 

W^'^{0,T), so up to a further extraction of a subsequence, it converges to some function U 
in L^(0, T). As a result, there exists a function ijj of the variable t such that Uh ^ u + i/j in 
L^{0,T-W^''^{T'^)). 

From the a priori estimate ()54p . the sequence {F{mh)) is bounded in LT((0,T) x T^) for some 
7 > 0, which implies that it is uniformly integrable on (0,T) x T^. On the other hand, F{mh) 
converges almost everywhere to F{m). Therefore, from Vitali's theorem, see e.g. |19j , F{mh) 
converges to F{m) in L^((0, T) x T^), (in fact, it is also possible to show that F{mh) converges 
to F{m) in L5((0,r) x T^) for all q € [1,7)). 

It is then possible to pass to the limit in the discrete Bellman equation, which yields that ^ = 
in the sense of distributions in (0, T). Hence if) is a, constant. 

We are left with proving that ijj is indeed 0. For that, we split into the sum + rjh, where 

• ^^h\t£{tn,tn+i\ is constant w.r.t. t and piecewise linear w.r.t. x, and takes the value 
i/(A/jw"'+^)jj at the node 

• rjh is the remainder, see (fTS]) . This term is constructed by interpolating the values F{m'^j) — 
g{xij, [D^u^^^jij) at the grid nodes. 

From the observations above, {r]h) converges in L^((0,T) X T2), (because of the strong conver- 
gence of S/ufi and of F{mh)). On the other hand, from it is not difficult to see that {^h) is 
a Cauchy sequence in L^(0, T; (PF'*'^/('^-^)(T2))') for s large enough, (here {W^^I^/^f^-^\T'^))' is 
the topological dual of W'^I^/^I^-^\T'^)). Hence, (^) converges in ^^(0, T; (VF^'^/('^-i)(T2))'). 
Therefore, converges in C'^([0, T]; (T^^''^/^^"^) (T^))'); since {uh{t = 0)) converges to uq, we 
see that ■0 = 0. 

This implies that the extracted sequence converges to u in L^(0, T; VF-'^'^(T^)). Since the 
limit is unique, the whole family {uh) converges to u in L^(0, T; T^^'^(T^)) as h and tend to 
0. ■ 

We give the corresponding theorem in the ergodic case, without proof, because it is quite similar 
to that of Theorem [5j 

Theorem 6 We make the standing assumptions stated at the beginning of § 15.^1 and we assume 
furthermore that there is unique classical solution {u,m,X) of such that m > and 

Jr^2 u{x)dx = 0. 

Let Uh (resp. m^) he the piecewise bilinear function in C(T^) obtained by interpolating the values 
Ui,j (resp mij) at the nodes o/T^, where {{uij), {mij),Xh) is the unique solution of the following 
system: 

for all < i,j < Nh, ruij > 0, 

-u{Ahu)ij + g{xij, [Dhu]ij) + Xh = F{mij), 

-i'{Ahm)ij - Tijiu,m) =0, (58) 

h'^T.^,jUiJ =0, h^EiJ^Uij =1. 

As h tends to 0, the functions Uh converge in W^''^{T'^) to u, the functions rrih converge to m 
in L2-''2(T2), and Xh tends to X. 
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A Proofs of some technical lemmas 

Proof of Lemma [2] For all r e , we have 

9q{x,q)-r = • (-lg,<ori,l52>0'^2,-lg3<0'^3,l<?4>0'^4) 

'^"^ {-pm + P2r2 - Psrs + P^ri) 



Hence, 

-gq{x,q) -{q-q) = -Gp{jp) ■ f-lgi<o('7i - lg2>o(g2 - ^2), -lg3<o(93 - gs), lg4>o(g4 

= -I^W~'^{-pi{qi - qi) +P2{q2 - 92) -pziq?, - qs) +P4(g4 - 94) )• 



But 



- ^i) = piiPi - Pi) - piqt < PiiPi - 

P2{q2 - q2) = P2{P2 - P2) - P2q2 < ^2(^2 - P2), 

-pz{q-i - qs) = P3(P3 - ps) - p?At < Pafe - Ps), 
^4(0^4 - qi) = PiiPi - Pa) - Piq^ < Pi{pi - Pi)- 



Therefore 



- gq{x, q)-{q-q)> -I3\pf '^p-{p-p) = -Gp{p) ■ {p - p), 
and (|27p follows immediately. 

Proof of Lemma [3] If /3 > 2, then 

9{x,q) - g{x,q) - gq{x,q) ■ {q - q) > G{p) - G{p) - Gp{p) ■ [p - p) 

= / (l- s)Gpp{sp + {I - s)p){p- p) ■ ip- p)ds 
Jo ^ 

> f3\p - p|2 / (1 - s)\sp + (1 - s)pf-'^ds, 



where the first (resp. second) inequality comes from Lemma[2] (resp. Lemma [T]). Hence, 



g{x,q)-g{x,q)-gg{x,q)-{q-q) > f^lpf'^lP - P\^ I - s)^~^ds 

= \pf~'\p-p\'- 

On the other hand. 



Jo 



-J^^\pf-'\p-p\'- 



g{x,q) - g(.x,q) - gq{x,q) ■ {q - q) > 



The last two estimates yield ()28p because 1 > j^^, then (|29p . 
If 1 < ^ < 2 and p + p^O, then 

9{x,q) - g{x,q) - gq{x,q) ■ {q - q) > / {1 - s)Gpp{sp + {1 - s)p){p - p) ■ {p - p)ds 



> /3(/3-l)b-p|2 l\l-s)\sp+{l-s)pf-^ds, 
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where the first (resp. second) inequahty comes from Lemma[2] (resp. Lemma[T]). But \sp+ (1 - 
s)p\ < 2max(|p|oo, \p\oo) and since P <2, \sp+{l - s)pf-^ > 2/^-2 min(|p|^ ^ \p\L Hence, 



g{x, q) - gix, q) - g,ix, q) ■ {q - q) > 2^-'' ^ " 1) mindpl^,-^^ \pf-'') \ (1 - s)ds, 
which yields (j30|) . 







Proof of Lemma [4] We have that 
{9q{x,q) -gq{x,q)) ■ r 
=/3 ( \pf~'^ {-piri + P2r2 - Psrs + p4r4) - \pf~'^ {-pm + P2?^2 - P3?'3 + P4?'4) ) • 



Call £ the function defined on (M+)^ by 

^(p) = /3|p|'^-2p.Diag(-l,l,-l,l)r, 

where Diag(— 1, 1, —1, 1) stands for the diagonal matrix in R^^'* whose diagonal is (—1, 1, —1, 1). 
We have 

{9q{x, q) - gq{x, q))-r = £{p) - i{p) = / ip{sp + (1 - s)p) ■ {p - p)ds, iip + p^O, 

Jo 

{gq{x,q) - gq{x,q)) ■ r = 0, iip + p = 0. 

But 

Ipip) = /3(/3 - 2)\pf-^ {p ■ Diag(-1, 1, -1, l)r) p + /3b|^-2Diag(-l, 1, -1, l)r, Vp / 0. 
Hence, if p + p ^ 0, 

{gq{x,q) -gq{x,q)) ■ r 

= - 2) ^ \sp + (1 - s)pf-^ [{sp + (1 - s)p) ■ Diag(-1, 1, -1, l)r) ({sp + (1 - s)p) ■ {p - 
+13 j \sp+{l-s)pf-^({p-p)-'D\a.g{-l,l,-l,l)r^ds. 

(60) 

Call / (respectively //) the first (respectively second) integral in ([5U|) . It is clear that 



|-?^| < |p — p| I?"! / \sp -\- {1 — s)p\^ 2 (is < max ( 



1/3-2 Ul/3-2 



Aw )\p-p\VV 



We also have 

|//| <max(H^-Mp|^~2)|^_^||^|^ 
and (1311) follows from the last two estimates. 
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